Software-Based Erasure Codes for Scalable Distributed Storage
نویسندگان
چکیده
This paper presents a new class of erasure codes, Lincoln Erasure codes (LEC), applicable to large-scale distributed storage that includes thousands of disks attached to multiple networks. A high-performance software implementation that demonstrates the capability to meet these anticipated requirements is described. A framework for evaluation of candidate codes was developed to support in-depth analysis. When compared with erasure codes based on the work of Reed-Solomon and Luby, tests indicate LEC has a higher throughput for encoding and decoding and lower probability of failure across a range of test conditions. Strategies are described for integration with storage-related hardware and
منابع مشابه
A Non-MDS Erasure Code Scheme for Storage Applications
This paper investigates the use of redundancy and self repairing against node failures indistributed storage systems using a novel non-MDS erasure code. In replication method, accessto one replication node is adequate to reconstruct a lost node, while in MDS erasure codedsystems which are optimal in terms of redundancy-reliability tradeoff, a single node failure isrepaired after recovering the ...
متن کاملApplied Erasure Coding in Networks and Distributed Storage
The amount of digital data is rapidly growing. There is an increasing use of a wide range of computer systems, from mobile devices to largescale data centers, and important for reliable operation of all computer systems is mitigating the occurrence and the impact of errors in digital data. The demand for new ultra-fast and highly reliable coding techniques for data at rest and for data in trans...
متن کاملHomomorphic Self-repairing Codes for Agile Maintenance of Distributed Storage Systems
Distributed data storage systems are essential to deal with the need to store massive volumes of data. In order to make such a system fault-tolerant, some form of redundancy becomes crucial. There are various overheads that are incurred due to such redundancy most prominent ones being overheads in terms of storage space and maintenance bandwidth requirements. Erasure codes provide a storage eff...
متن کاملOn the Speedup of Recovery in Large - Scale Erasure - Coded Storage Systems ( Supplementary File )
Our work focuses on the recovery solutions for XORbased erasure codes. We point out that regenerating codes [5] have recently been proposed to minimize the recovery bandwidth in distributed storage systems. The idea is that surviving storage nodes compute and transmit linear combinations of their stored data during failure recovery. On the other hand, in XOR-based erasure codes, we do not requi...
متن کاملErasure Coding in Distributed Storage Systems
Data centers are nowadays highly distributed, possibly over continents, to cope with the huge amounts of data collected these days. To deal with data loss in an environment where hardware failures are very common, researchers have developed new kinds of erasure codes. Erasure coding is a common technique used in all sorts of digital communication, including deep space communication and QR-codes...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2003